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Abstract 

A mechanism for releasing information about a statistical database with sensitive data must 
resolve a trade-off between utility and privacy. Publishing fully accurate information maximizes 
utility while minimizing privacy, while publishing random noise accomplishes the opposite. Pri- 
vacy can be rigorously quantified using the framework of differential privacy, which requires 
that a mechanism's output distribution is nearly the same whether or not a given database row 
is included or excluded. The goal of this paper is strong and general utility guarantees, subject 
to differential privacy. 

We pursue mechanisms that guarantee near-optimal utility to every potential user, indepen- 
dent of its side information (modeled as a prior distribution over query results) and preferences 
(modeled via a loss function). Our main result is: for each fixed count query and differential 
privacy level, there is a geometric mechanism M* — a discrete variant of the simple and well- 
studied Laplace mechanism — that is simultaneously expected loss-minimizing for every possible 
user, subject to the differential privacy constraint. This is an extremely strong utility guarantee: 
every potential user u, no matter what its side information and preferences, derives as much 
utility from M* as from interacting with a differentially private mechanism M„ that is optimally 
tailored to u. More precisely, for every user u there is an optimal mechanism for it that 
factors into a user- independent part (the geometric mechanism M*) followed by user-specific 
post-processing that can be delegated to the user itself. 

The first part of our proof of this result characterizes the optimal differentially private 
mechanism for a fixed but arbitrary user in terms of a certain basic feasible solution to a linear 
program with constraints that encode differential privacy. The second part shows that all of 
the relevant vertices of this polytope (ranging over all possible users) are derivable from the 
geometric mechanism via suitable remappings of its range. 

1 Introduction 

Organizations including the census bureau, medical establishments, and Internet companies collect 
and publish statistical information [6l |19] . The census bureau may, for instance, pubhsh the result 
of a query such as: "How many individuals have incomes that exceed $100,000?". An implicit 
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hope in this approach is that aggregate information is sufficiently anonymous so as not to breach 
the privacy of any individual. Unfortunately, publication schemes initially thought to be "private" 
have succumbed to privacy attacks [IHllTTlllj, highlighting the urgent need for mechanisms that 
are provably private. The differential privacy literature [lOl El HU UHl [U [3, US Hj has proposed a 
rigorous and quantifiable definition of privacy, as well as provably privacy-preserving mechanisms 
for diverse applications including statistical queries, machine learning, and pricing. Informally, for 
a G [0, 1], a randomized mechanism is a-differentially private if changing a row of the underlying 
database — the data of a single individual — changes the probability of every mechanism output by 
at most an a factor. Larger values of a correspond to greater levels of privacy. Differential privacy 
is typically achieved by adding noise that scales with a. While it is trivially possible to achieve any 
level of differential privacy, for instance by always returning random noise, this completely defeats 
the original purpose of providing useful information. On the other hand, returning fully accurate 
results can lead to privacy disclosures [8J. The goal of this paper is to identify, for each a £ [0, 1], 
the optimal (i.e., utility-maximizing) a- differentially private mechanism. 



2.1 Differential Privacy 

We consider databases with n rows drawn from a finite domain D. Every row corresponds to an 
individual. Two databases are neighbors if they coincide in n — 1 rows. A count query f takes a 
database d G as input and returns the result f{d) £ N = {0, . . . , n} that is the number of rows 
that satisfy a fixed, non-trivial predicate on the domain D. Such queries are also called predicate 
or subset-sum queries; they have been extensively studied in their own right [3 HJ [121 [5] , form 
a basic primitive from which more complex queries can be constructed [4]. 



A randomized mechanism with a (countable) range i? is a function x from Z?" to R, where x^r 
is the probability of outputting the response r when the underlying database is d. For a G [0, 1], 
a mechanism x is a- differentially private if the ratio x^^^jXfi^T lies in the interval [a,l/a] for 
every possible output r £ R and pair d\^d2 of neighboring database^]. (We interpret 0/0 as 1.) 
Intuitively, the probability of every response of the privacy mechanism — and hence the probability 
of a successful privacy attack following an interaction with the mechanism — is, up to a controllable 
a factor, independent of whether a given user "opts in" or "opts out" of the database. 

A mechanism is oblivious if, for all r £ R, Xd^j. = x^^r whenever f{di) = f{d2) — if the 
output distribution depends only on the query result. Most of this paper considers only oblivious 
mechanisms; for optimal privacy mechanism design, this is without loss of generality in a precise 
sense (see Section 16. 2p . The notation and definitions above simplify for oblivious mechanisms 
and count queries. We can specify an oblivious mechanism via the probabilities Xi^ of outputting 
a response r £ R for each query result i G N; a-differential privacy is then equivalent to the 
constraint that the ratios Xir/x^i^iy lie in the interval [a, 1/a] for every possible output r £ R and 
query result i £ N \ {n}. 

Example 2.1 (Geometric Mechanism) The a- ^eomeiric mec/ianism is defined as follows. When 
the true query result is f{d), the mechanism outputs f{d) + Z. Z is a random variable distributed 

^ The usual definition of differential privacy requires this bound on the ratio for all subsets of the range. With a 
countable range, the two definitions are equivalent. 
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as a two-sided geometric distribution: Pr[Z = z] = j^a'^' for every integer z. This (obHvious) 
mechanism is a-differentially private because the probabihties of adjacent points in its range differ 
by an a factor and because the true answer to a count query differs by at most one on neighboring 
databases. 

2.2 Utility Model 

This paper pursues strong and general utility guarantees. Just as differential privacy guarantees 
protection against every potential attacker, independent of its side information, we seek mechanisms 
that guarantee near-optimal utility to every potential user, independent of its side information and 
preferences. 

We now formally define preferences and side information. We model the preferences of a user 
via a loss function I; l{i,r) denotes the user's loss when the query result is i and the mechanism's 
(perturbed) output is r. We allow / to be arbitrary, subject only to being nonnegative, and nonde- 
creasing in \i — r\ for each fixed i. For example, the loss function \i — r\ measures mean error, the 
implicit measure of (dis) utility in most previous literature on differential privacy. Two among the 
many other natural possibilities are (i — r)^, which essentially measures variance of the error; and 
the binary loss function hinihf), defined as if z = r and 1 otherwise. 

We model the side information of a user as a prior probability distribution {pi} over the query 
results i £ N. This prior represents the beliefs of the user, which might stem from other information 
sources, previous interactions with the mechanism, introspection, or common sense. We emphasize 
that we are not introducing priors to weaken the definition of differential privacy; we use the 
standard definition of differential privacy (which makes no assumptions about the side information 
of an attacker) and use a prior only to discuss the utility of a (differentially private) mechanism to 
a potential user. 

Now consider a user with a prior {pi} and loss function / and an oblivious mechanism x with 
range R. For a given input d with query result i = f{d), the user's expected loss is X^rei? ' ^(^' '^)' 
where the expectation is over the coin flips internal to the mechanism. The user's prior then yields 
a measure of the mechanism's overall (dis)utility to the the user: 

^Pi^Xir ■ l{i,r). (1) 

ieN r&R 

This is simply the expected loss over the coin tosses of the mechanism and the priorH We can 
then define the optimal a-differentially private mechanism for a user as one that minimizes the 
user-specific objective function ([T|). 

2.3 User Post-Processing 

Could a single mechanism be good simultaneously for all users? A crucial observation for an 
affirmative answer is that a user has the power to post-process the output of a privacy mechanism, 
and that such post-processing can decrease the user's expected loss. 

■^The central theorem of choice theory (e.g. [151 Chapter 6]) states that every preference relation over mechanisms 
that satisfies reasonable axioms (encoding "rationality") can be modeled via expected utility, just as we propose. In 
particular, this theorem justifies the use of priors for expressing a rational user's trade-off over possible query results. 
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Figure 1: Theorem 13.11 For every rational user u, (a) can be factored into a user- independent 
part (the a-geometric mechanism) followed by a user-dependent post-processing step (the optimal 
remap y"). 



Example 2.2 (Post-Processing Decreases Loss) Fix a database size n that is odd. Consider 
a user with the binary loss function and prior pQ = Pn = ^f"^, Pj = for all j G \ {0, n}, that 
interacts with the a-geometric mechanism. Without post-processing, i.e., when the user accepts 
the mechanism's outputs at face value, the user's expected loss is (2 • a)/(l + a). If the user maps 
outputs of the geometric mechanism that are at least (n + l)/2 to n and all other outputs to 0, it 
effectively induces a new mechanism with the much smaller expected loss of a^"''^^^^'^ /{I + a). 

In general, a (randomized) remap of a mechanism x with range is a probabilistic function y, 
with Ur'r denoting the probability that the user reinterprets the mechanism's response r' £ R as 
the response r € R. A mechanism x and a remap y together induce a new (a-differentially private) 
mechanism y o x with (y o x)ir = X^r'ei?. " 2/r'r- 

We assume that a (rational) user with prior p and loss function /, interacting with a publicly 
known mechanism x, employs a remap y that induces the mechanism y o x that minimizes its 
expected loss ([I]) over all such remaps. It is well known (e.g. [El Chapter 9]) and easy to see 
that, among all possible (randomized) remappings, the optimal one follows from applying Bayes 
rule and then minimizing expected loss. Precisely, for each response r of x, compute the posterior 
probability over query results: For every i G R, p{i\r) = {xir ■ Vi) / (Ylii'^R^Vr 'Pi')- Then, choose 
the query result i* £ R that minimizes expected loss subject to the posterior and set yri* = 1 and 
yri = for i ^ i* . This remap is simple, deterministic, and can be computed efficiently. 

3 Main Result and Discussion 

Our main result is that for every a £ [0, 1], the a-geometric mechanism is simultaneously optimal 
for every rational user. 

Theorem 3.1 (Main Result) Let x^ denote the a- geometric mechanism for some database size 
n > 1 and privacy level a G [0, 1], and let y^ denote an optimal remap of x'-' for the user u with 
prior p and (monotone) loss function I. Then y" o x'^ minimizes u's expected loss ([T]) over all 
oblivious, a -differentially private mechanisms with range N . 



4 



This is an extremely strong utility-maximization guarantee: every potential user u, no matter 
what its side information and preferences, derives as much utility from the geometric mechanism 
as it does from interacting with a differentially private mechanism M„ that is optimally tailored 
to u. We reiterate that the prior from the utility model plays no role in the definition of privacy, 
which is the standard, worst-case (over adversaries with arbitrary side-information and intent) 
guarantee provided by differential privacy. We emphasize that while the geometric mechanism 
is user- independent (all users see the same distribution over responses), different users remap its 
responses in different ways, as informed by their individual prior distributions and loss functions. 
Rephrasing Theorem 13.11 for every user there is an optimal mechanism for it that factors into a 
user-independent part — the a-geometric mechanism — and a user-specific computation that can be 
delegated to the user. (See Figure El) 

Theorem 13. II shows how to achieve the same utility as a user-specific optimal mechanism with- 
out directly implementing one. Direct user-specific optimization would clearly involve several chal- 
lenges. First, it would require advance knowledge or elicitation of user preferences, which we expect 
is impractical in most applications. And even if a mechanism was privy to the various preferences 
of its users, it would effectively need to answer the same query in different ways for different users, 
which in turn degrades its differential privacy guarantee. 

In Theorem 13. H the restriction to oblivious mechanisms is, in a precise sense, without loss of 
generality. (See Section [6^ ) The restriction to the range N effectively requires that the mechanism 
output is a legitimate query result for some database; this type of property is called "consistency" 
in the literature (e.g. [2j). 

4 Related Work 

Differential privacy is motivated in part by the provable impossibility of absolute privacy against 
attackers with arbitrary side information [8]. One interpretation of differential privacy is: no matter 
what prior distribution over databases a potential attacker has, its posterior after interacting with 
a differentially private mechanism is almost independent of whether a given user "opted in" or 
"opted out" of the database [l^. Below we discuss the papers in the differential privacy 
literature closest to the present work; see [9] for a recent, thorough survey of the state of the field. 

Dinur and Nissim [7j showed that for a database with n rows, answering O(nlog^n) randomly 
chosen subset count queries with o{^/n) error allows an adversary to reconstruct most of the rows 
of the database (a blatant privacy breach); see Dwork et al. [TT] for a more robust impossibility 
result of the same type. Most of the differential privacy literature circumvents these impossibility 
results by focusing on interactive models where a mechanism supplies answers to only a sub-linear 
(in n) number of queries. Count queries (e.g. [3 [12]) and more general queries (e.g. [lOllIl]) have 
been studied from this perspective. 

Blum et al. [5] take a different approach by restricting attention to count queries that lie in a 
restricted class; they obtain non-interactive mechanisms that provide simultaneous good accuracy 
(in terms of worst-case error) for all count queries from a class with polynomial VC dimension. 
Kasiviswanathan et al. [13j give further results for privately learning hypotheses from a given class. 

The use of abstract "utility functions" in McSherry and Talwar [16] has a similar flavor to 
our use of loss functions, though the motivations and goals of their work and ours are unrelated. 
Motivated by pricing problems, McSherry and Talwar I16| design differentially private mechanisms 
for queries that can have very different values on neighboring databases (unlike count queries); 
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they do not consider users with side information (i.e., priors) and do not formulate a notion of 
mechanism optimahty (simultaneous or otherwise). 

Finally, in recent and independent work, McSherry and Talwar (personal communication, Oc- 
tober 2008) also apply linear programming theory in the analysis of privacy mechanisms. Again, 
their goal is different: they do not consider a general utility model, but instead ask how expected 
error must scale with the number of queries answered by a differentially private mechanism. 



5 Proof of Main Result 

This section proves Theorem 13.11 The proof has three high-level steps. 

1. For a given user u, we formulate the problem of determining the differentially private mech- 
anism that minimizes expected loss as a solution to a linear program (LP). The objective 
function of this LP is user-specific, but the feasible region is not. 

2. We identify several necessary conditions met by every privacy mechanism that is optimal for 
some user. 

3. For every privacy mechanism that satisfies these conditions, we construct a remap y such 
that yo x'^ = X. By assumption, a rational user employs an "optimal remap" of x*^, so the 
mechanism induced by this map must be optimal for the user u. 

Fix a database size n and a privacy level a. Theorem 13.11 is trivially true for the degenerate 
cases of a = 0, 1. So, we assume that a £ (0, 1). For every fixed user with loss function I and prior 
p, the formulation of privacy constraints in Section 12.11 together with the objective function ([T]) 
yields the following LP whose solution is an optimal mechanism for this user. 



User-specific LP: 

minimize "^^pi"^^ Xir ■ l{i,r) (2) 

ieN rGiV 

Xir - a- a;(i+i)r > Vr G A*" \ {n}, Vi G iV (3) 

a-Xir~ a;(i+i)r < Vr G \ {n}, Vi G iV (4) 

J2^^r^l \/ieN (5) 

Xir>o VieNyreN (6) 



Since the LP is bounded and feasible, we have the following (e.g. [3]). 

Lemma 5.1 Every user-specific LP has an optimal solution that is a vertex. 

For the rest of this section, fix a user with prior {pi} and a loss function l(i, r) that is monotone 
in |« — r| for every i £ N. Fix a mechanism x that is optimal for this user, and also a vertex of 
the polytope of the user-specific LP. Vertices can be uniquely identified by the set of all constraints 
that are tight at the vertex. This motivates us to characterize the state of constraints (slack or 
tight) of mechanisms that are optimal for some user. 

We now begin the second step of the proof. We will view x as a (n -|- 1) x (n -|- l)-matrix where 
rows correspond to query results (inputs) and columns correspond to query responses (outputs). 
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Figure 2: The database size n is 5. Figure shows an optimal 1/2-differentiany private mechanism 
for a user with prior 1/4, 0, 1/4, 0, 1/4, 1/4 on the six possible results and the loss function r) = 
\i - rP-^ 
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Figure 3: The constraint matrix for the mechanism from Figure [2j 

We state our necessary conditions in terms of an n x (n + 1) constraint matrix C associated with the 
mechanism x. Row i of the constraint matrix corresponds to rows i and i + 1 of the corresponding 
mechanism. Every entry of C{i,r), for i£N\n, r£N takes on exactly one of four values. 
If Xir = = then C{i,r) = Z. If Xir,X(^i^iy ^ Z, then there are three possibilities. If 

a ■ Xir = then C{i,r) =J,. If Xir = a ■ X(^i_^_iy then C{i,r) =|. Otherwise C{i,r) = S. 

Example 5.2 Figure [2] shows an optimal, 1/2-differentially private mechanism for a specific user. 
Figure [3] lists the constraint matrix of this mechanism. This mechanism can be derived from the 
1/2-geometric mechanism by mapping every negative number to 0, every number larger than 5 to 
5, 1 to 2 and the other numbers to themselves. 

The constraint matrix is well defined: since a < 1, at most one of a ■ Xij = or Xij = 

a ■ holds (or else both are zero). Also, using that a > 0, Xij = implies that Xkj = for all 

k. Thus every column of C is either all Z's, which we then call a Z-column, or has no Z entries, 
which we then call a non-Z column. 

By definition, the constraint matrix encodes the state of all the privacy constraints ^ and 
The constraint matrix also implicitly encodes the state of the inequality constraints ([6]). First, 
Xij > if and only if the jth column of C is non-Z. Also, we can assume that Xij < 1 for all 
i,j £ N; otherwise, since a > 0, x has singleton support and the proof of the theorem is trivial. 
Since vertices are uniquely identified by the set of all the constraints that are tight at the vertex, 
we have the following. 

Observation 5.3 Every mechanism that is a vertex of the polytope of the user-specific LP has a 
unique constraint matrix. 

Let s denote the number of S entries of C. Let Sj denote the number of S entries in the ith 
non-Z column. Let Si = X]fc<i^«- ^ denote the number of Z-columns of C. The next few 
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lemmas, Lemma 15. 4| 15.51 and 15. 6| use the fact that x is an optimal mechanism to show that the 
rows of C must exhibit certain structure. Corollary 15.71 and Lemma 15.81 additionally use that x 
is a vertex to show that the total number of S entries of C is equal to the number of Z-columns. 
Unless otherwise stated, we ignore Z -columns of C. The following lemma holds for every feasible 
mechanism. 

Lemma 5.4 No row of the constraint matrix is either all [ 's or all | 's. 

Proof: Suppose row i of tliG constraint matrix is ctll J,'s. So, for all t G A^, * ^^^r — •^(i-\-i)r 

and 

because a < 1, Xir > Since "^j-Xir = 1, J2r^{i+i)r < 1) contradicting the feasibility of the 

mechanism. The other case is analogous. □ 

The next, key lemma relies on the monotonicity of the loss function I and the optimality of x. 

Lemma 5.5 Every row of the constraint matrix C has the following pattern: some | 's followed by 
at most one S followed by some j 's. 

Proof: Suppose not; then there exists a row index i and column indices / < m such that C{i, I) 
and C{i,m) There are two cases: i < {I + m)/2 and i > {I + m)/2. We prove the first case, 
and suggest modifications to the proof for the second. We now define a feasible mechanism x' with 
strictly smaller expected loss, contradicting the optimality of x. We multiply the numbers Xj/^ for 
all i' < i by a 1 — (5 factor for some small 6 > and set Xj/; = Xi'i + 6 ■ Xi^m- 

Because i < (/ + m)/2, \i' — l\ < \i' — m\, and, so, for all i' < i the expected loss strictly decreases 
for strictly monotone loss functions and priors with support A^H We now discuss feasibility x'. 
Notice that we modify only two columns of x. The set of constraints dS]) are preserved by feasibility 
of X and the definition of x'. Because a > 0, Xi'm > for all i' € N and for sufficiently small 6 the set 
of constraints ([6]) continue to hold. For all j < i, privacy constraints that involve the numbers {xjm} 
continue to hold as all we are doing is scaling. For all j < i, privacy constraints that only involve 
the numbers in the sets {xjm} and {xji} continue to hold since differential privacy is preserved by 
scaling and adding. Finally, the privacy constraints involving the numbers Xim, xi^i^i-^^^, xu, X(i^i^i 
are preserved for sufficiently small 5 since C{i,l) and C{i,m) T^j. 

For the second case, when i > (/ + m)/2, we interchange the roles of the columns / and m and 
modify rows i' > i + 1 rather than i' < i, i.e., we multiply Xi'i for alH' > i + 1 by 1 — 5 for a small, 
positive 5 and set Xj'm = iCi'm + S ■ xm. The proof can be modified accordingly. □ 

The next lemma relates adjacent rows of C; it uses the previous lemma combined with the fact 
that each row of x is a valid probability distribution. 

Lemma 5.6 For all i £ ... n — 2, row i + 1 of C has at least one more I than row i of C; unless 
row i + 1 has a S in which case row i + 1 has at least as many | 's as row i. 

Proof: Suppose to the contrary that the pair of rows i,i + 1 violates the condition in the lemma 
statement, then we will show that row i + 2 ol the mechanism violates the probability constraint ([5]), 

Recall the pattern of a row i of C from Lemma 15.51 Suppose that row i has j's in exactly 
positions ... j — 1. Let J2k<j — ^) ^ij — ^i J2k>j ^ik = c and X(j_|_i)j = b'. Because row i of x 

■^If the prior does not have full support or if the loss function is not strictly monotone, we can reach the same 
conclusion using perturbations and a limiting argument. 
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must satisfy the probability constraint ([5]), we have 

a + b + c = l. (7) 

By assumption, C{i,k) =[, for all k < j. By Lemma [5.51 we have that for all A; G j + 1 . . . n, 
C{i,k) =t and C{i,j) is either a 5 or a j. So, Z]fc<j = ct ■ a, and Z^fc>j ^(j+i)^ = c/a. 

Because row i + 1 of x must satisfy the probability constraint ([5]) , we have 

a ■ a + b' + c/a = 1. (8) 

By assumption the rows + l violate the lemma condition. So, by Lemma [5.5l and the definition of 
the index j, we have C(i + 1, k) =] for all columns k > j. So, J2k>j ^{i+2)k = 1/a Ylk>j = 
b'/a + c/a^. Suppose that for each k G — 1, a;(j_|_2)fc is as small as possible subject to 

the privacy constraint (so X(j+2)A; = • this is the worst case for our argument. Then, 

Sfc<j ^(i+2)fc = X]fc<j ^(i+i)fe = CK^ • a. To complete the proof we show that a^a + b' /a + cjc? is 
strictly larger than 1 and so row i + 2 of x violates the probability constraint. 

Using Equations ([7|) and ([8]) to eliminate a and c, we can show that c?a + b' /a + cjo? = 
1/a + a — l + b — b'a, which is at least 1/a + a — 1 because b — b'a > (x is a-differentially private) . 
Simple algebra shows that l/a + o — 1>1 whenever (q — 1)^ > 0, which holds since a < 1. □ 

The next two lemmas relate the number of Z-columns of C to the number of S entries in it. 

Corollary 5.7 The number s of S entries of C is at least the number z of Z -columns of C . 

Proof: Suppose there are a |'s and b S"s in the first row of the constraint matrix. From 
Lemma |5.4| the first row cannot consist entirely of fs, i.e. o + 6 > 1. By Lemma 15.61 the number 
of I's in the last row is at least a + (n — 1) — (s — 6). i.e. at least n — s. By Lemma 15.41 the number 
of I's in the last row must be at most n — z. Chaining the two inequalities gives us the result. □ 

Unlike the previous lemmas, the next lemma uses the fact that x is a vertex. 

Lemma 5.8 The number s of S entries of C is equal to the number z of Z -columns of C . 

Proof: By Corollary 15.71 all we need to show is that s < z. Recall that x is the solution to an 
LP with (n + 1)'^ -variables. Since x is a vertex of the polytope of the user-specific LP, it must be 
at the intersection of at least (n + 1)^ linearly independent constraints. 

Let us account for them. The n + 1 constraints ^ are tight. Exactly z{n + 1) constraints of 
the type specified by ^ are tight — these involve variables Xjj, for all i € N, such that j is an index 
of some Z-column of C. Thus, of the remaining privacy constraints, at least (n + l)(n — z) must 
be tight. Because (by definition of C) every such tight constraint corresponds to a unique i or t 
entry, there are at least (n + l)(n — z) such entries. Thus, of the n{n + 1 — z) entries which are not 
Z's, at most n(n -\- 1 — z) — (n -\- l)(n — z) = z oi the entries are S. □ 

Lemma [5.91 leverages the conditions on the rows of C established by previous lemmas to establish 
conditions on the columns of C via a counting argument. This completes the second step of the 
proof of Theorem 13.11 

Lemma 5.9 For all i £ . . . n — z, the ith non-Z column of C has | 's in rows indexed ... i — 1 + 
Si-i, S 's in the Si positions that follow and | 's in the remaining positions. 
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Proof: We do induction on non-Z columns. The base case: The first column does not contain 
any |'s, because then by Lemma [5.51 an entire row contains only |'s, contradicting Lemma [5.41 By 
Lemma 15.61 and Lemma 15.51 a S cannot follow a | and a | cannot follow a S" in the same column. 
Therefore, the first column contains sq 5's, followed by j's. 

For the inductive step, assume column i — 1 has |'s and S"s in rows . . . i—2+Si-i and so column 
i has 1 's in these positions by Lemma [531 We now show by contradiction that C(i — 1 + Sj-i , i) =1 ; 
assume otherwise. Let the indicator variable / be 1 if C(i — H-S'i_i, i) =| and if C(i — H-5j_i, i) = 
S. By the induction hypothesis, C{i — 1 + Si-i,i — 1) =| and so by Lemma [5?5l row i — 1 + Si^i 
has at least i + I j's (we index from 0). By Lemma |5.5| every one of the rows from row i + S'j-i 
to row n — 1 must either contain an additional | or expend one of the s — Si-i — 1 + / remaining 
5"s. Thus row n has at least i + I + n — i — s + 1 — I = n + 1 — s [''s. By Corollary 15.81 s = z and 
so the last row of C is all j's, which contradicts Lemma |5.4[ 

We next show that C{i + Si-i,i) ^] and so with an argument similar to the base case, rows 
i + Si-i ... n of column i consists of Sj zeros, followed by |'s, concluding the proof of the induction 
step. Suppose C{i + Si-i,i) =t. By the induction hypothesis, C{i — 1 + Si-i,i — 1) =J,, thus row 
i — 1 + Si-i contains at least i j's. By induction C{i + Si-i,i — 1) =|. So by Lemma [5.5[ row 
i + Si-i does not contain a S and contains i |'s; this contradicts Lemma 15.61 □ 

This concludes the second step of the proof. For the third step of the proof, we work with a 
finite-range version of the geometric mechanism x*^, called G. G is wox^ , where wis a. deterministic 
remap that maps all negative integers to zero, all integers larger than n to n, and all other integers 
to themselves. We show, constructively, that there exists a remap y" such that x = y" o x*^. This 
proves the theorem because the composed remap y^ow applied to x'^ induces x: x = {y^ ow)ox'^^ 
Lemma 15.101 clarifies the structure of columns of mechanisms induced by deterministic remaps of 
G. Lemma 15.111 combines Lemma 15.101 and Lemma 15.91 to prove Theorem 13.11 

Lemma 5.10 Consider a deterministic map that maps the integer sequence a. . .b (and no others) 
to a fixed I € N. Then, the Ith column of the constraint matrix of the mechanism induced by applying 
this map to G has | 's in rows 0, . . . , a — 1, S 's in rows a, . . . ,b — 1, and | 's in the remaining rows. 

Proof: Let the variables {gij} denote the entries of G and let Wi be the entry of in row i of 
column / of the induced mechanism. By definition, Wi = Yl'j=a9ij- 

The jth column of G is a multiple of the column vector (a^ , a^~^, . . . , a^, . . . , a"'~^~^, a""-'). 
So, for < i < a — 1, a < j < b, gij = a(7(j_|_i)j, and so, wi = au)j+i. Similarly, for i, j, 
b <i <n, a < j <b gij = {l/a)gi+ij, and so, Wi = (l/a)u;i+i. 

Finally, for every i, a < i < b — 1, there exist a < < b, such that gij = agi+ij 

and gij' = {l/a)gi^ij/ . Also a < I. Therefore, for i such that a < i < b — 1 we have that 
awi < Wj+i < {l/a)wi. We have the proof by the definition of constraint matrices. □ 

The following lemma shows that there exists a remapping of G that induces a mechanism with 
constraint matrix identical to C, the constraint matrix corresponding to our fixed (vertex) optimal 
mechanism x. By Observation 15.31 the induced mechanism must be x and the theorem is proved. 

Lemma 5.11 There exists a remapping of G such that the induced mechanism has a constraint 
matrix identical to C . 

^Incidentally, there do exist vertices of the polytope of the user-specific LP that are not derivable from G by 
remapping. Fortunately, these vertices are not optimal for any user. 
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Proof: Define the following (user-specific) deterministic map y, which we apply to G. Let ki be 
the index of the rth non-Z column of C . Map the integers in i + Si-i .. .i + Si to ki. We check that 
y is well defined: Si + 1 responses of G are mapped to ki, and so, a total ofn + 1 — z + s distinct 
responses are mapped. By Lemma l5.8( 2: = s, so the map is well defined for every member of the 
range N G. 

No integer that is an index of a Z-column of G is in the range of the map. So, the constraint 
matrix of the induced mechanism has the same set of Z-columns as G. By Lemma 15.91 the ith 
non-Z column has fs in rows 0, . . . , i — 1 + 5's in rows i + Si-i, . . . ,i — 1 + Si, and |'s in 

the remaining rows. By Lemma 15.101 and the definition of y, this is precisely the pattern of the ith 
non-Z column of the constraint matrix of the induced mechanism. □ 



6 Discussion 
6.1 Uniqueness 

Are there other mechanisms for which an analog of Theorem 13.11 holds? An obvious candidate 
is the well-studied Laplace mechanism (release the result with noise added from a distribution 
with density function e/2 • e"*^'*', where a = e""^), essentially a continuous version of the geometric 
mechanism. Here is an example where the Laplace mechanism compares unfavorably with the 
geometric mechanism. Fix a count query, a database size n, and a user with loss function lerr 
and prior (1/2,1/2) over the two possible results {0,1}. The geometric mechanism has (optimal) 
expected loss a/(l+a) whereas the Laplace mechanism has expected loss yfoLjl. The approximation 
achieved by the Laplace mechanism tends to 00 as a approaches 0. Though somewhat pathological, 
this example rules out the Laplace mechanism as an answer to the above question. 

So, is the a-geometric mechanism the unique mechanism for which an analog of Theorem 13.11 
holds? No, because, the proofs from the previous section demonstrate that the range-restricted 
variant G also satisfies the guarantee of the theorem. But a uniqueness result is possible: if 
we restrict attention to mechanisms with range N , then G is the unique simultaneously optimal 
mechanism, up to a permutation of the range. 

Theorem 6.1 Fix a database size n > 1 and a privacy level a £ [0,1]. Suppose there is an a- 
differentially private mechanism x with range N such that there exists remap y^ for every user u 
where y'^ ox minimizes the expected loss of user u (Eq\^ over all oblivious, a- differentially private 
mechanisms with range N . Then, there exists permutation remap p such that po x = x^ . 

Proof: Consider a user with uniform prior over and the loss function l^in- We can solve the 
user-specific LP to show that G is the unique optimal mechanism for the user, where the user is 
constrained to accept the mechanism's responses at face value. Suppose there is a mechanism G' 
distinct from G that is optimal for all users, and in particular for this user. As G in the above 
sense, G' must induce G on application of the user's optimal remap. By assumption, the range of 
G' is N, while G is onto N. Recall that optimal remaps are deterministic and so G' must also be 
onto N and the optimal remap must be a permutation. □ 
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6.2 Obliviousness 



Recall that our main result (Theorem l3.ip compares the utility of the remapped a-geometric mech- 
anism only to oblivious mechanisms. While natural mechanisms (such as the Laplace mechanism 
from pro]) are usually oblivious, we now justify this restriction from first principles. Suppose we 
deploy non-oblivious mechanisms. Measuring the expected utility of such a mechanism for a given 
user requires the user to have a prior over databases, and not merely over query results. If a user 
begins only with a prior over query results (arguably the most natural assumption if that's all it 
cares about), the prior could be extended to one over databases in numerous ways. Singling out 
any one extension would be arbitrary, so we consider optimizing worst-case utility over all such 
extensions. Formally, for Lemma l6 . 2 1 b elow . we temporarily replace ^ with the objective function 

max pdy^Xdr-l{fid),r). (9) 

In Q , — > Pi indicates that pd is a prior over D" that induces the user's prior pi over A^, meaning 
Pi = Yld\f(d)=iPd ^"-"^ every i € N. The following lemma then shows that the restriction to oblivious 
mechanisms is without of generality. 

Proposition 6.2 Fix a database size n > 1 and privacy level a. For every user with prior {pi} 
and loss function I, there is an a -differentially private mechanism that minimizes the objective 
function Q and is also oblivious. 

Proof: Fix a privacy level a, a database size n and a user with prior {pt} and monotone loss 
function /. Let a; be a mechanism that minimizes ([9]) for this user. We define a mechanism x' that is 
oblivious, a-differentially private, and has at most as much expected loss as x for the objective ([9]). 
For any database d £ define E{d) as the set of databases with the same query result as d. 
For a database d G D" and response r £ N, let x'^^ be the average of Xd'r over the databases 
{d'\d' G E{d)}; x' is oblivious by definition and also a valid mechanism, in that it specifies a 
distribution over responses for every underlying database. 

We now show that x' is a-differentially private. Fix two databases di,d2 G -D" such that di and 
d2 differ in exactly one row; We need to show that ax'^^^. < x'^^^.. Assume /(di) 7^ 7(^2), otherwise 
the proof is trivial. 

For any database of E{di), we can generate all its neighbors (databases that differ in exactly 
one row) in E{d2) by enumerating all the ways in which we can change the query result by exactly 
1. For instance when f{di) = f{d2) + 1, pick one of the n — f{di) rows that satisfy the predicate 
in di and change its value to one of those that violates the predicate. This process is identical for 
all databases of E{di), and so for all d G E[di), the number of neighbors of d that belong to the 
set E{d2) is the same (does not vary with d). Similarly, for all d G £'(^2), the number of neighbors 
of d that belong to the set E{di) is the same. 

Consider the following set of inequalities that hold because x is a-differentially private: d G 
E{di), d' G E{d2), where di and ^2 are neighbors, axdr < Xd'r- By the argument in the above 
paragraph, all the databases in E{di) appears equally frequently in the left-hand-side of the above 
inequality and all the databases in E{d2) equally frequently in the right-hand-side. Summing the 
inequalities and recalling the definition of x' completes the proof of privacy. 

We now show that x' has better worst case expected loss. Since x' is oblivious, the expected 
loss is the same for all prior distributions over databases that induces the prior {p,} over results. 
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Figure 4: The mechanism xi 

By definition of x', the expected loss is: YlieNPi '^'^9d\f{d)=i X^rGAf ^<^^' ' ^(^' '^)- -^^'^ mechanism x, 
we can construct an adversarial distribution over databases that induces {pj}, such that for every 
i E N, the weight pi is assigned entirely to a database d that maximizes " Kf{d),r) over 

= i. Thus x incurs at least as much expected loss as x'. □ 

An alternative is to model users as having a prior over databases. Though priors on databases 
induce priors on query results, the converse is not necessarily true. Formally, for Theorem 16.31 
below, we use the following objective function. 

PdY.'^dr-l{f{d),r). (10) 

We show that no analog of Theorem 13.11 is possible in this modelH 

Theorem 6.3 There exists a database size, level of privacy and two users, each with monotone 
loss functions and distinct priors over databases such that no mechanism is simultaneously optimal 
for both of them. 

We now prove Theorem 16. 3i Suppose that the domain D is {0, 1} and we draw databases from 
D^. Fix a count query that counts the number of rows that are 1. Fix a privacy level a = 1/2. We 
label databases by the set of rows that have values 1; in this notation, the result of the query is the 
cardinality of the set. The first user's prior over databases is \ on {1} and {2}, \ - e on {2,3}, e 
on {1,3} and on all others. The second user's prior is defined by interchanging the roles of rows 
1 and 2 in this definition. Both users have the monotone loss function l{i,r) = \i — r\^~^ . There 
exist small, positive e and 5, for which the unique (non-oblivious) optimal mechanism (where the 
user accepts results at face value) for the first user can be shown to be the mechanism xi defined 
in Figure [621 (Only the rows that referred to by our proof are specified.) Similarly, the unique 
optimal mechanism for the second user can be derived by interchanging the roles of the rows 1 and 
2. 

We will show that there is no 1/2-differentially private mechanism x that implements xi and X2 
simultaneously. Assume to the contrary: Let yi and y2 be deterministic maps that when applied 

''Proposition 16.21 implicitly shows that with the restriction to oblivious mechanisms, Theorem 13.11 would continue 
to hold in this model. However, Theorem 16.31 implies that obliviousness is no longer without loss of generality. 
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Figure 5: The mechanism M 



to X induce xi and X2 respectively. (We skip the easy modification to the proof that allows yi and 
2/2 to be randomized.) We first show that x, yi and y2 have the following form: 

Lemma 6.4 Without any loss of generality, x has a range R = {/,m, n,o} of size 4, yi rnaps I 
and n to 1 and m and o to 2 and y2 maps I and o to 1 and m and n to 2. 

Proof: Both xi and X2 have range size 2. Label each member of the range of x in one of four 
ways based on where it is mapped by yi (1 or 2) and 1/2 (1 or 2). All the range points with the 
same label may be combined into one range point. For instance o consists of all the range points 
mapped to 2 by yi and 1 by 2/2- The proof is complete. □ 
We are now ready to prove Theorem 16. 3[ 

Proof of Theorem 1 6. 31 " By the previous lemma and the definitions of xi and X2, the mechanism x 
must have the form in Figure [6^21 Because x is 1/2-differentially private and as {1} and {2} differ 
in exactly two rows, columns n and o yield the following inequalities: 3 + (5i < 4(^2; and 3 + (^2 ^ 4(5i. 
Adding the two inequalities, we have: 6 < 2>{5i + 82)- Because all the entries of x are probabilities, 
we have < (5i,(52 < 1- So it must be that (5i = (52 = 1. By a similar argument applied to the 
databases {3} and {4}, we can show that 5-^ = 84^ = 1. 

Thus, the probability masses on / when the underlying databases are {1} and {1,3} are 7/12 
and 1/6 respectively. But this violates privacy because, the two databases differ in exactly one row 
but the two probabilities are not within a factor 2 of each other. □ 

7 Future Directions 

We proposed a model of user utility, where users are parametrized by a prior (modeling side infor- 
mation) and a loss function (modeling preferences). Theorem 13.11 shows that for every fixed count 
query, database size, and level of privacy, there is a single simple mechanism that is simultaneously 
optimal for all rational users. Are analogous results possible for other definitions of privacy, such as 
the additive variant of differential privacy (see 0)? Is an analogous result possible for other types 
of queries or for multiple queries at once? When users have priors over databases (Theorem 16. 3p . 
are any positive results (such as simultaneous approximation) achievable via a single mechanism? 



14 



8 Acknowledgments 



We thank Preston McAfee, John C. Mitchell, Rajeev Motwani, David Pennock and the anonymous 
referees. 

References 

[1] L. Backstrom, C. Dwork, and J. Kleinberg. Wherefore art thou r3579x?: anonymized social 
networks, hidden patterns, and structural steganography. In Proceedings of the 16th Interna- 
tional Conference on World Wide Web (WWW), pages 181-190, 2007. 

[2] B. Barak, K. Chaudhuri, C. Dwork, S. Kale, F. McSherry, and K. Talwar. Privacy, accuracy, 
and consistency too: a holistic solution to contingency table release. In Proceedings of the 26th 
ACM SICACT-SIGMOD-SIGART Symposium on Principles of Database Systems (PODS), 
pages 273-282, 2007. 

[3] D. Bertsimas and J. N. Tsitsiklis. Introduction to Linear Optimization. Athena Scientific, 
1997. 

[4] A. Blum, C. Dwork, F. McSherry, and K. Nissim. Practical privacy: The SuLQ framework. 
In Proceedings of the 24th ACM SIGACT-SICMOD-SIGART Symposium on Principles of 
Database Systems (PODS), pages 128-138, 2005. 

[5] A. Blum, K. Ligett, and A. Roth. A learning theory approach to non-interactive database 
privacy. In Proceedings of the 40th Annual ACM Symposium on Theory of Computing (STOC), 
pages 609-618, 2008. 

[6] U. S. Census Bureau. The 2008 statistical abstract. 

|http:/ /www.census.gov/compendia/statab/ 

[7] I. Dinur and Nissim K. Revealing information while preserving privacy. In Proceedings of 
the 22nd ACM SICACT-SIGMOD-SIGART Symposium on Principles of Database Systems 
(PODS), pages 202-210, 2003. 

[8] C. Dwork. Differential privacy. In Proceedings of the 33rd Annual International Colloquium on 
Automata, Languages, and Programming (ICALP), volume 4051 of Lecture Notes in Computer 
Science, pages 1-12, 2006. 

[9] C. Dwork. Differential privacy: A survey of results. In 5th International Conference on 
Theory and Applications of Models of Computation (TAMC), volume 4978 of Lecture Notes in 
Computer Science, pages 1-19, 2008. 

[10] C. Dwork, F. McSherry, K. Nissim, and A. Smith. Calibrating noise to sensitivity in private 
data analysis. In Third Theory of Cryptography Conference (TCC), volume 3876 of Lecture 
Notes in Computer Science, pages 265-284, 2006. 

[11] C. Dwork, F. McSherry, and K. Talwar. The price of privacy and the limits of LP decoding. 
In Proceedings of the 39th Annual ACM Symposium on Theory of Computing (STOC), pages 
85-94, 2007. 



15 



[12] C. Dwork and K. Nissim. Privacy-preserving datamining on vertically partitioned databases. 
In 24th Annual International Cryptology Conference ( CRYPTO), volume 3152 of Lecture Notes 
in Computer Science, pages 528-544, 2004. 

[13] S. P. Kasiviswanathan, H. K. Lee, K. Nissim, S. Raskhodnikova, and A. Smith. What can 
we learn privately? In Proceedings of the 49th Annual IEEE Symposium on Foundations of 
Computer Science (FOCS), pages 531-540, 2008. 

[14] S. P. Kasiviswanathan and A. Smith. A note on differential privacy: Defining resistance to 
arbitrary side information, http://arxiv.org/abs/0803.3946vl, 2008. 

[15] A. Mas-Colell, M. D. Whinston, and J. R. Green. Microeconomic Theory. Oxford University 
Press, New York, 1995. 

[16] F. McSherry and K. Talwar. Mechanism design via differential privacy. In Proceedings of the 
48th Annual IEEE Symposium on Foundations of Computer Science (FOCS), pages 94-103, 
2007. 

[17] A. Narayanan and V. Shmatikov. Robust de-anonymization of large sparse datasets. In 
Proceedings of the 2008 IEEE Symposium on Security and Privacy (SP), pages 111-125, 2008. 

[18] K. Nissim, S. Raskhodnikova, and A. Smith. Smooth sensitivity and sampling in private 
data analysis. In Proceedings of the 39th Annual ACM Symposium on Theory of Computing 
(STOC), pages 75-84, 2007. 



[19] Wikipedia. AOL search data scandal. http://en.wikipedia.org/wiki/AOL_search_data_scandal 



16 



